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ABSTRACT 



This paper summarizes plans to implement portfolio 
assessment for graduate students in the Instructional Technology program in 
the College of Education at the University of Houston (Texas) . Specific 
advantages of portfolios, and electronic and World Wide Web formats in 
particular, hold promise for the development and assessment of the student 
population. The goals for implementation are outlined, including a plan for 
informational seminars, criteria for item selection, portfolio components, 
anticipated barriers, and the role that portfolios are intended to play in 
the student graduation process. Contains 14 references. (Author/MES) 
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Abstract: Ibis paper summarizes oiir initial plans to implement portfolio assessment for 
graduate studqits in our Instructional Teclmology (IT) program. Specific advantages of 
portfolios, and electronic and Web formats in particular, hold promise fiir the 
development and assessment of our student population. We outline the goals we have for 
in^lementation that include a plan fi>r infi)nnational seminars, criteria for item selection, 
portfolio conqxments, antic^ted barriers, and the role we intend portfolios to play in the 
graduation process of our students. 



Introduction 



Performance-based portfolios are gaining acceptance as a viable alternative form of assessment in 
educational settings. Teacher education programs, in particular, have found portfolios to be an elastic 
assessment format c£^>able of addressing criteria not a part of traditional assessments, such as continuous 
student reflection, individual assessmsat of growth and change, iterative evaluation of learning goals, and 
the contextual examination of created prodhicts in relation to complex teaching processes ^arton & 
Collins, 1993; Gmllaume & Yopp, 1995; Levin, 1996; Richards, 1998; Snyder, Lippincott, & Bower, 1998; 
Wade & Yarbrough, 1996). Portfolio assessment allows for the development of pedagogy and discipline- 
oriented philosophy (Carroll, PotthofiF, & Huber, 1996), as well as the evolution and e 7 q>ression of a 
professional voice (McKinney, 1998). The distinct advantage of portfolios that appeals nmst to us, as 
faculty in an Instructional Technology program, is that the ownership of learning and product production 
shifts away fix>m the teacher (Wiggins, 1989). Instep it reflects a jointly negotiated process, reached by 
the teacher and the student, that entails detailed discussions and customized evaluation of learning, hi 
conqriling a portfolio, the student selects multiple samples of work for the purpose of demonstrating 
personal competence in various focets of learning required by the program. 

In an Instructional Technology (TI^ p rogr am , essentially all artifacts of learning, whether products 
that demonstrate skill or reflection, are by definition digital products. Thus, we assert tiiat as a storage and 
display medium, electronic portfolios in particular woitid provide a natural solution to representing student 
learning in Instructional Technology. Philosophically, we adhere to the notions of constructivism, and whh 
regards to the assessment of digital products, we subscribe specifically to "constructivism" as eTqxessed by 
Resnick (1 998) as the idea that people construct new knowledge with particular effectiveness when they are 
engaged in constructing pasanaUy-meaningful products. We argue that the processes of learning, 
implementation, and production common in IT programs can be best represented tWu^ the use of digital 
portfolios. Further, the emergence of the Web as a predominant learning environment in education and 
business led us to consider this particular medium as the portal to student portfolios. The Web format 
encourages individuality by allowing students to demonstrate creativity in presentation and organization, to 
create multiple design formats for multiple audiences (Watkins, 199Q, and to easily establish connections 
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among related portfolio components (McKinney, 1998). The Web, as both a technology and an interface, 
enables the student ultimate control in assembling and ease of reK)rganizmg, as well as the ability to 
integrate narratiye captions among the learning evidence to emphasize the interrelated nature of the 
learning (Watkins, 1996). The Web environment permits students the flexibility to maintain their portfolios 
in a Web-space that can be remotely accessed from anywhere at any time, by the student, faculty, peers, 
and potential employers. Finally, Web-based portfolios promote seamless access to student work by 
( eliminating software and platform incompatibilities encountered when viewing electronic portfolios created 

! with multiple authoring tools (Mills, 1997). 



Our Context 

The Instructional Technology program in the College of Education at the University of Houston 
offers both Masters and Doctoral degrees to primarily part-time, commuting students who hold full-time 
jobs in both education and industry. Assessment strategies currently vary among our courses depending on 
the course content, structure, and individual instructor preference. Digital design and multime^a projects 
are typically submitted on disk or iq>Ioaded to a specific course server location, while traditional reports 
and papers are turned in either electronically or on paper. As students progress through our program, we 
currently have no consistent standard for collecting and comparing the work they produce in separate 
courses, therefore there is no easy way to drawany type of comparisons of program-wi^ effectiveness. 

While much of the literature on the use of portfolios describes implementation within preservice 
teacher education, our faculty sees the promise of a portfolio assessment strategy for our graduate program, 
for many of the same reasons. First and foremost, we value and encourage student ownership of learning, 
and we see the compilation of portfolios as the ideal instrument to coach students to observe and manage 
their own learning progress. We believe we can use portfolios to accomplish this goal not only because 
their creation helps make explicit the connections among seemingly discrete but realistically related 
courses, but also because portfolios will assist our students in valuing the course projects in relation to our 
larger program goals. We want to e liminat e the prevalent idea t^ course products are produced in 
isolation merely to get a grade, destined only to collect dust on faculty shelves. Portfolios would allow our 
Acuity the opportunity for more meaningful and interactive feedback to guide our students toward learning 
goals, dialogue which would ultimately allow students to originate and hone their individual professional 
voices within the diverse IT field. 

We see Web-based portfolios, in particular, as the most versatile format for incorporating the multiple 
projects that our students produce using a variety of software. A key advantage for our commuting, and 
frequently telecommuting, population of students and faculty, is that Web-based portfolios could be revised 
and assessed fixmi any computer at any time. We envision portfolio assessment as initially supplementing, 
and potentially replacing, our written comprehensive exams as a graduation requirement Such a change 
would better reflect our constructivist teaching and learning philosophy, as well as the project-based nature 
of our program. Finally, we look forward to the ongoing feedback on the effectiveness of our program that 
portfolios would provide our faculty (McKiimey, 1998; Snyder, Lippincott, & Bower, 1998). Such 
feedback, we believe, could directly feed the continuous improvement and development of our program. 

Digital portfolios for our IT program seem to be the natural format whose time has come, both for our 
shidents and for us. Other plans we are considering for portfolios include c£^rturing the work examples to 
provide our students with examples of exen^lary models produced by their peers and archiving these 
products in an organization scheme that benefits the larger education community. The driving force behind 
our interest in pcatfolios, however, is the progressive and comprehensive review of the projects, process, 
and learning across courses and throughout the entire experience of the student in the program. 



Initial Plans for Incorporating Portfolio Assessment 

In planning for a portfolio strategy in the IT program, we began by asking questions relating to the 
nature of the knowledge, application, and synthesis level skills that students will encounter in the program. 
What information do we want students graduating from the IT program to know, and what skills should 
they be able to demonstrate? What kinds of evideiK:e will verify that the information and skills have been 



ERIC 



3 



1118 



learned? How elfective is our program at preparing future instructional technologists, and what could we 
do differently to give our students an even more meaningful preparation in instructional technology? These 
questions are aligned with portfolio development criteria chscussed by others (Barton & CoUins, 1993; 
Georgi & Crowe, 1998; Snyder, Lippincott, & Bower, 1998). 

With those over-arching questions in mind, our portfolio process will begin with a, required seminar 
at the beginning of the semester for entering Masters-seeking students, allowing ample time to assimilate 
the concept of using portfolios before courses have begun (Stone, 1998). In this seminar, we will orient 
students to the ideas of "digital portfolios," articulate flexible criteria that will provide guidelines for the 
individual portfolio requirements, and enable them to think of the program and the learning process as one 
that is evaluated by more than the individual instructor. By presenting these criteria in the form of 
organized rubrics, students wiU be made aware of expectations we have for the demonstration of 
proficiency and skill, thus de-mystifying the evaluation process and allowing students to take an early and 
active role in the planning for and management of their own learning during the course of the program. 

Instruction on technical considerations and style guidelines for the Web-based portfolio will be 
presented in an effort to ensure ease of accessibility of student work throughout the graduate experience. 
Upon entering our program, each student wiU be given permanOTt disk-space on the department server to 
be used for the duration of their studies. To assist with ^e details of file-management, we will establish a 
consistent hierarchy of file organization within each student’s Web iiKlex that is based on our course 
numbering system. Sufficient hands-on time for practicing such procedures as file transfer and organization 
will be given at the seminar so that students walk away with a clear understanding of portfolio maintenance 
basics. 

The seminar will conclude by addressing the actual work of compiling portfolio components. 
Substantial time will be spent demonstrating strategies for selecting portfolio items to represent a range of 
work quality and type, with a focus on ethnogr^hic methods of coUecting and analyzing data. By 
addressing portfolios with specific research and analysis methods, we will empower each student with the 
skills necessary to use the data to develop an individual theory of instruction^ technology as it relates to 
his or her own learning artifacts (Snyder, Lippincott, & Bower, 1998). Specific exanq)les will be given to 
students to illustrate the process of authoring reflections on their work, or ^sustained hard personal looks at 
oneself and one’s practice” (p. 54). Final thoughts in this first s eminar will involve es tablishing the focus of 
each student’s portfolio through the writing of an iiKlividual goal statement 

Once established, the portfolio procedures will be regularly reinforced throughout the individual 
courses in the FT program, allowing us to design and adhere to standard policy and a consistent technical 
format for all of the various courses. We anticipate that this advanced and sustained attention to the 
portfolio construction, maintenance, and presentation will contribute to the overall success of the evaluation 
procedure. 



The Balance among Utility, Reflection, and Creativity 

Our FT portfolios will be useful only if they transcend a merely pleasing display of conq)leted wcnk to 
instead make conq)elling arguments of each student’s knowledge and skill. At the same time, we hope that 
the portfolios will highlight interrelations among instructional themy, research, and practice, both intended 
and serendipitous, as evidenced in the products of learning and the ability of each student to articulate these 
ideas. 

Despite our intention to provide a consistent overriding portfolio fiamework, we do not believe that 
the requirements of included work should be entirely prescribed Such demands for conformity would 
counter our goals of reflection, individuality, and custcxmized feedback, and might, in fact, preclude student 
creativity and individuality, thus resulting in artificially s imil ar products fixjm all students (Mills, 1997). 
Like others (Scanlan & Heiden, 1996; Snyder, Lippincott, & Bower, 1998), we recognize Ae ne^ for a 
convergence of perspectives in the design of our portfolio process so that our portfolios can be useful for 
both summative purposes and individual student reflectioiL Therefore, we will strive for a balance between 
student-selected items chosen to illustrate personal learning and items suggested by the faculty to 
demonstrate learning in accordance with program goals. Consideration of item selection will be encouraged 
regularly at the conclusion of each semester, and items previously selected may be re-evaluated in 
comparison with newer projects. Students will compose narrative cqrtions to rationalize the inclusion of 
each item in the portfolio, thus constructing a unique comparative thread linking skills and projects. 
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Student-written captions will enable faculty to evaluate learning products with the students' voices 
articulating the reasoning behind projects (Barton & Collins, 1993). In addition to formal narratives, 
students will have the opportunity to include personal reflections about the progress toward attaining their 
learning goals. 

Considered along with the selected work examples and various written commentaries, the Web 
interface and navigational sequence will serve as the ultimate demonstration of creative design and 
systematic development expected of IT graduates. The final portfolio component will be a summative 
“epilogue” written by each student to br^g closure to his or her graduate experience. This culminating 
document will address questions specifically posed by the faculty that are intended to guide the student to 
synthesize his or her understanding of theory and practice. 

Successful completion of the portfolio process in our program will be marked by a ^*portfolio 
consultation” to occur during the last semester of the student’s program and involve the participation of 
peers and faculty advisors. Consultations will take the tone of a collaborative forum, a collective “ think 
tank’’ of instructional technology issues led by the portfolio author, as opposed to the generally one-sided, 
directed nature of an academic inquisition, a hurdle commonly seen as part of a graduation defense. This 
collegial approach will allow us to further capitalize on the portfolio as a tool for the continued growth of 
both students and faculty. Such examination of learning evidence and subsequent reflection, we argue, is 
not possible in our existing traditional written comprehaisive exam format (Bali, Wright, & Foster, 1997; 
Barton & Collins, 1993). 



Challenges and Future Directions for Portfolios 

As our portfolio evaluation process matures, we foresee related research investigations, such as 
examining the impact of portfolio development on student learning and reflection, the benefits of increased 
student and faculty collaboration, the extent to which portfolios allow us to assess the validity of course 
objectives, and the role of the information gained through the process on ongoing improvement of course- 
to-program goal ahgnment For now, portfolio assessment is scheduled to commence in our Instruction 
Technology program with the incoming Master’s students in the Spring semester of 2000. We understand, 
however, that the best-laid plans will not ensure flawless implementatioa The literature reviewed thus far 
has been clear about the potential challenges we may face by incorporating portfolios into our progranL For 
example, we anticipate that ensuring gr adin g consistency with this alternative assessment strategy can 
prove problematic, as can establishing the expectation that students honestly reflect on their learning rather 
than report back only what they think we want to hear (Scanlan & Heiden, 1996). We concur with the 
recommendation of Scanlan & Heiden that developing an evaluation rubric will address this issue by 
explicitly communicating our evaluation criteria to students early in the program. Genuine faculty 
participation with and commitment to the benefits of portfolio assessment, consistently communicated to 
students, should also invite more honest student participation. Another barrier we expect is the additional 
time commitment requir^ fiom both students and faculty advisors to assemble the portfolios concurrently 
with fulfilling project requirements for individual courses (McKinney, 1998). Our goal of consistently 
addressing portfolio format and content as a component of each of our courses will, we believe, make the 
necessary time commitment more manageable and ultimately worthwhile. Our exploration of possible 
solutions to these and other perceived and anticipated obstacles will continue. We are convinced, however, 
that the authentic assessment and related benefits possible through the use of Web-based portfolios are 
much too advantageous to ignore. 
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